UNITED STATES PATENT APPLICATION 

of 

JUERGEN VOGEL 
THOMAS SAGCOB 

GERD MAUTHE 
MICHAEL MAIER 
GUENTHERHUBER 
KARL BUEHLER 
and 

BERNHARD BROGHAMMER 
for 

VIDEO SIGNAL PROCESSING SYSTEM 



VIDEO SIGNAL PROCESSING SYSTEM 



BACKGROUND OF THE INVENTION 

The invention relates to a video signal processing system, and in particular to a system for 
5 processing both video data and graphic data to provide a mixed output signal. 

Video signal processing systems often include a filter unit that converts video pictures 
from a first format to a second format. For example, the first format may be coded in accordance 
with the digital studio standard of the International Telecommunications Union UIT-R (or of the 
Comite Consultatif International des Radiocommunications: CCIR) 601. An example of 
10 converting from a first format to a second format is the conversion from a PAL 16x9 picture 
format with 720 pixels per line (P/L) and 576 lines (L), into the National Television System 
Committee (NTSC) 16x9 picture format, with 480 pixels per line and 480 lines. 

In addition, video signal processing systems often include a computing unit for mixing 
video and graphic pictures (e.g., layering and alpha-blending). The computing unit for mixing 
15 may weight different pictures by adjustable factors (i.e., alpha blending), so that transparent 
pictures can be displayed by combining and superposing the images (i.e., layering). For example, 
transparent buttons may be inserted while the original video picture continues to be visible in the 
background. 

At this time, such systems exist only as individual components. A disadvantage of 
20 combining such systems is the amount of memory required to filter and mix video and graphic 
pictures. 

Therefore, there is a need for a memory efficient video signal processing system capable 
of converting video signals from a first format to second format, and mixing several pictures. 



-2- 



SUMMARY OF THE INVENTION 

Briefly, according to an aspect of the present invention, a video signal processing system 
for processing a video data V JN and graphic data D^ p includes a filter unit, which receives the 

5 video data V lN . The filter unit filters the video data V m to convert the video data V IN into video 
pictures formated with a different number of columns and/or lines, and provides a filtered video 
signal indicative thereof. The filter unit buffers individual pixels and/or lines in a first memory 
device. A second memory device receives and stores the graphic data D^ p and the filtered video 

signal and provides stored signals indicative thereof. A third memory device is connected to the 
10 second memory, and stores data received from the second memory device. A mixing unit 
receives and mixes the stored graphic data and the stored filtered video data to provide a video 
output signal V OUT . 

In one embodiment, the video signal processing system may be implemented with a 
programmable logic module (e.g., a FPGA) and additional memory units. For example, the first 
15 and third memories may be implemented as RAM external to the programmable logic module, 
while the second memory is located on the programmable logic module and configured as a 
cache to provide fast access. The remaining components of the video signal processing system 
may be located on the programmable logic module. 

The graphic data may include bitmaps that are provided by a microprocessor. In this way, 
20 for example, transparent buttons may be inserted, while the original video picture associated with 
the video data continues to be visible in the background. 
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The system may also include a controller that controls the processing of the video and 
graphic signals, and specifically the programmable logic module and the memory components. 
The controller may be located within the programmable logic module. The controller may 
include a microprocessor, or receive instructions from an external microprocessor. The 
5 microprocessor may control parts of the program execution or the entire execution of the digital 
video signal processing program. 

The video signal processing system preferably operates in real time. 

In an alternative embodiment, the video signal processing system for processing a video 
data V m and graphic data D mP , comprising a horizontal filter that receives the a video data V /N 

10 and converts the video data V JN into video pictures formatted with a different number of 
columns, and provides a horizontally filtered video signal indicative thereof, wherein the 
horizontal filter buffers individual pixels and/or lines in a first memory device. A second 
memory device receives and stores the graphic data D mP and the filtered video signal, and 

provides stored signals indicative thereof. A third memory device is connected to the second 
15 memory, and stores data received from the second memory device. A mixing and filtering unit 
receives the stored graphic data and the stored horizontally filtered video data, and vertically 
filters the stored horizontally filtered video data to convert the video data into video pictures with 
a different number of lines. The mixing and filtering unit provides a vertically filtered signal 
indicative thereof and mixes the stored graphic data with the vertically filtered video signal to 
20 provide a video output signal V OUT . 

Significanlty, in this alternative embodiment, the vertical filter is no longer implemented at 
the input in the filter block, but at the output of the so-called layering block. This makes an 
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additional filter block memory at the signal/video signal input unnecessary since the additional 

existing memory can be used at the same time. 

In a preferred embodiment, the first memory is configured as a fast cache memory and the 

second memory as working memory in the form of a random access memory. 
5 The system of the present invention may be used for an interlace progressive conversion. 

This is needed for future displays with higher resolution. In principle, the same saving applies 

there, mainly of a CPU, which is situated directly before the vertical filter of the first 

arrangement, and of an external memory. The memory saving in this case is even much greater, 

since the previous technique requires an additional half-picture memory for the corresponding 
10 signal processing. For example, in the case of the PAL standard, two lines must be interim 

stored in an external RAM. The second embodiment described herein may perform the interlace 

progressive version with the existing CPU and memory. 

These and other objects, features and advantages of the present invention will become 

more apparent in light of the following detailed description of preferred embodiments thereof, as 
15 illustrated in the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWING 

FIG. 1 is a block diagram illustration of a first embodiment of a video signal processing 
system; and 

20 FIG. 2 is a block digram illustration of a second embodiment of a video signal processing 

system. 

DETAILED DESCRIPTION OF THE INVENTION 
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FIG. 1 is a block diagram illustration of a first embodiment of a video signal processing 
system, which receives a video signal input V m , and a graphic input signal D^ p . The video 

signal input V IN is connected to a filter unit that includes a horizontal filter 1 1 and a vertical filter 
12. An external memory 17 (preferably RAM) is associated with the filter unit. The filter unit 
5 provides a filtered video signal on a line 30 to an intermediate memory 13, which also receives a 
the graphic input signal D^ p . The intermediate memory 13 is connected to another external 
memory 16. 

The intermediate memory 13 is also connected to a mixing unit 15, which provides a 
video output signal V 0UT . A control unit 14 controls the filter unit (i.e., the horizontal filter 11 

10 and the vertical filter 12) the memories 13, 16, and 17, and the mixing unit 15. In a preferred 
embodiment, the horizontal filter 11, vertical filter 12, intermediate memory 13 (cache), mixing 
unit 15, and control system 14 are integrated into a programmable logic module (e.g., a FPGA). 

The digital video data of the video input signal V IN are scaled by the horizontal filter 1 1 
and the vertical filter 12 (e.g., the data are converted into another size relationship to match a 

15 display device). For horizontal filtering, generally only a few pixels from the same line are 
needed. Since these succeed one another directly in time, only one memory is typically required 
for a few pixels. However, in the vertical filter 12, the pixels of several lines (e.g., of at least two 
lines) must be calculated together. Thus the information of at least one line must be buffered in a 
memory device. This buffering requires a memory size of at least 710 pixels for the luminous 

20 density signal and of at least 360 pixels for each color difference signal. Accordingly, a total of 
1440 pixels (i.e., 720 pixels + (2x360) pixels) have to be stored. 
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The data are then written via the intermediate memory 13 (e.g., a cache), into an external 
memory 16 (e.g., RAM). In parallel with this, the graphic data (e.g., bitmaps) for example from a 
microprocessor (not shown), are written via the fast intermediate memory 13 into the external 
memory 16. 

5 These pictures are combined with one another in the mixing unit 15 (layering block). 

This is done in such a way that different pictures are calculated together by an adjustable factor, 
for example, using a technique often referred to as alpha-blending. The result is transparent 
pictures that are presented for display on a display device (e.g., a CRT or flat panel display). 

The system 10 is preferably operated in real time, which requires that the clock rate of the 
10 control system 14 be much higher than the clock rate of the input and output signal. 

FIG. 2 is a block digram illustration of an alternative embodiment video signal processing 
system. The system of FIG. 2 receives and processes the video input signal V IN , and the graphic 
input signal D^ p . Specifically, the video input signal V m is input to a horizontal filter 21 that 
provides a horizontal filtered signal on a line 32. The graphic input signal D^ p and the 

15 horizontal filtered signal on the line 32 are input to an intermediate memory 23. Similar to the 
first embodiment set forth in FIG. 1, the intermediate memory 23 is connected to an external 
memory 26. Furthermore, the intermediate memory 23 is connected to a mixing unit 25 and a 
vertical filter 22, which are preferably realized in one block as a mixing and filtering unit. The 
mixing and filtering unit provides a video output signal V OUT . The components (i.e., the 

20 horizontal filter, intermediate memory 23, external memory 26, and the mixing and filter units 22 
and 25) are connected via control lines to a control system 24. The control system 24, as well as 
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all the other components except the external memory 26, are preferably integrated into a 
programmable logic module 20 (e.g., a FPGA). 

In the embodiment illustrated in FIG. 2, the vertical filter 22 is located at the output of the 
mixing unit 25 (i.e., layering block), rather than at the input of the filter block as illustrated in 
5 FIG. 1. Significantly, in the embodiment illustrated in FIG. 2, the external memory 17 is omitted 
from the filter block since the existing external memory 26 (RAM) may be used at the same time. 

The hardware of the mixing unit 25 (layering CPU) may be identical to that of the first 
embodiment. Only the control unit 24 now manages the correct calculation of the output data 
through an appropriately modified program execution. The integrated use of the mixing unit 25 
10 (layering CPU) for vertical filtering is dynamically controlled by the control system 24 and/or by 
external instructions from a microprocessor. 

This utilizes the higher clock rate of the CPU as compared to the clock rate of the input 
and output data. This also makes it possible for the various picture data, which are to be 
recalculated, to be read out correspondingly fast from the external memory 26 without damaging 
15 their bandwidth. In comparison to the first embodiment, the processing resources for the vertical 
filter 1 1 and the additional external memory 17 are also saved. 

If this configuration is used for interlace progressive conversion, the memory saving is 
even greater. With interlace progressive conversion, a full picture with 576 lines is calculated by 
the vertical filter 22 from two half pictures, each with 288 lines. In this case, therefore, it must 
20 be possible to put a half picture, i.e. 288 lines, into intermediate storage. In arrangement 1, this 
must be possible both in the external memory 17 and in the external memory 16. In arrangement 
2, it is sufficient to buffer in the external memory 26. 
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In this arrangement the video data associated with the video input signal V [N are 
converted by the horizontal filter 21 into a video picture with a different number of columns. For 
this, about 5-6 pixels are be buffered in the external memory 26. The recalculated video data, 
together with the graphic data conducted over the graphic input signal D mP , are written via the 

5 intermediate memory 23 into the external memory 26. To superpose two pictures using the 
mixing unit 25, the video data of a complete full picture (and the graphic data) must be stored. 
Consequently, it is desirable if both the mixing and vertical filtering are performed in a single 
block of the mixing and filtering device 22, 25. 

Although the present invention has been shown and described with respect to several 
10 preferred embodiments thereof, various changes, omissions and additions to the form and detail 
thereof, may be made therein, without departing from the spirit and scope of the invention. 
What is claimed is: 
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